Skip to main content

Integrating Guardrails

Policies from DynamoGuard can be integrated with applications using two methods: Analyze and Managed Inference. These methods provide flexible options for integrated policies tailored to your use case.

Analyze Mode​

The Analyze method enables lightweight guardraling by evaluating the compliance of a piece of text. This can be useful for integrating DynamoGuard into systems with multiple LLM calls, evaluating corpuses of text, or performing one-off compliance checks.

How it Works​

  1. Send a request to the /analyze/ endpoint with the text to be analyzed and the policies to be applied
  2. DynamoGuard evaluates the text against the configured policies
  3. The response contains the analysis results, which can be used to determine the appropriate action, such as blocking non-compliant user inputs or model responses

Managed Inference​

Managed inference refers to the process of using DynamoGuard as a proxy for inferencing a third-party LLM. It provides end-to-end policy enforcement on both user inputs and model responses.

How it Works​

  1. Send a request to the /chat/ endpoint with the user input
  2. DynamoGuard evaluates the user input against the configured policies
  3. Based on the results, the input may be blocked, sanitized and forwarded to the configured LLM, or sent as-is to the LLM
  4. The processed input is sent to the configured LLM, which generates a response
  5. DynamoGuard evaluates the model response against the configured policies
  6. Based on the results, the response may be blocked, sanitized and returned to the user, or returned as-is to the user

Supported Model Providers​

Managed inference is currently supported for the following model API providers:

  • OpenAI
  • Azure OpenAI
  • Databricks
  • AWS Bedrock
  • Mistral

Managed inference is also supported for custom endpoints and providers. For detailed guidance on setting up managed inference for your solution, please reach out to the DynamoAI team.

Token Limits​

DynamoGuard is currently configured to guardrail up to 4096 tokens in a single analyze or managed inference call. If your product requires guardrailing higher token lengths, please reach out to our team.